Tell the model what counts as checking its work - #100

Closes #93.

In the dahua session, the model marked "Test the fix by running the app" as done after only py_compile and an import. It changed the window's layout several times without seeing it.

What changes

• System prompt: a short "Verifying your work" section replaces the two old bullets about verifying ("at least a syntax check" and "Never claim something works…"). It says:
  • a syntax check or an import only shows that code loads, so run what you changed the way it will be used;
  • to fix a bug, reproduce each problem the user reports first, then run the same checks to show they're gone;
  • check what you can't see (a GUI, a web page, a layout) without a screen: QT_QPA_PLATFORM=offscreen, a headless browser, or a script that prints sizes and positions or renders an image. The image part says "you look at with view_image" only when a vision model is available, so the model isn't told about a tool it doesn't have;
  • never claim something works, mark a todo done, or call it tested without having run it; say what you couldn't check, and what the user should look for.
• lcode bench --tasks runtime-bug (lcode.bench_runtime): a camera wall whose window grows with every video frame and can't be made smaller again.
  • The cause is the dahua bug, in plain Python: the minimum size comes from the frame, and the frame is scaled to the whole cell, status bar included.
  • Reading the code suggests band-aids. Fixing only the scaling, or ignoring the minimum sizes, fails part of the hidden test; the test checks this for each band-aid.
  • It isn't in the default set, like find-concept.
• Docs: how-it-works ("Checking its own work"), bench, and the CHANGELOG.

Measured (qwen3.6-35b, 32K, 12 GB laptop)

To be honest about it:

• No pass-rate change. The guidance doesn't change the pass rate on runtime-bug; the task is too hard for this model either way.
• What did change. With the new prompt, the model ran simulate.py first in every finished reasoning-on round. It reported the measured window size instead of claiming success: "stays at a stable size (2304×1031)".
• What didn't. It still left the second complaint, that the window can't be made smaller, unchecked.
• No regression. The default tasks are unchanged.
• Why ship it anyway. The guidance is cheap (about 200 tokens), and the task stays as a harder benchmark for stronger models.

🤖 Generated with Claude Code